Amir Globerson
Wednesday 20th July 2016
Time: 4.00pm
Ground Floor Seminar Room
25 Howland Street, London, W1T 4JG
Kernels for deep learning - with and without tricks
Neural networks have recently re-emerged as a powerful hypothesis class,
yielding impressive empirical performance in multiple domains. However,
their training is a non-convex optimization problem which poses exciting
theoretical and practical challenges. Here we argue that by extending
the class of neural nets, one can obtain a convex learning problem,
whose practical solution relies on the evaluation of a particular kernel
(i.e., the kernel ''trick''). We show that in some cases this kernel can
be calculated in closed form. We next turn to the case where the kernel
cannot be evaluated in closed form, and introduce a sampling based
algorithm for learning with the same hypothesis class. Our regret based
analysis shows that the sample complexity of the sampling algorithm is
similar to that of an algorithm that uses the exact kernel. Empirical
evaluation shows that the method is competitive with other kernels and
sampling based algorithms.